Does it have any implications on memory use or runtime file loading?

For at_runtime LoRA mode, I believe it wouldn't change at all.

For immediately LoRA mode, it could mean higher memory usage: currently, the code could be unloading the weights right after applying, since they wouldn't be needed anymore (need to check the code to be sure). And to change the weights, we need them back in memory, either reloading from disk or keeping them around in RAM. Generation latency would also increase a bit, because we'd need to reapply the LoRAs (but only when the weight is changed).

Personally I have seen this request a few times. There is demand for it. If its a bit slower during a switch that is better than not having it at all. Just make sure nothing changes if its not used.

Got a first somewhat-working version.

I've included code for the <lora:name:weight> syntax on the prompt, to make testing easier. The API code is implemented, but I didn't test it yet.

As suspected, immediately LoRA mode discards the weights as soon as they are applied (lora->free_params_buffer() in apply_loras_immediately). So we need to either remove that call (as I've done for now), or restrict changing LoRA weights to the at_runtime mode. One way to do that without an extra command-line flag could be:

• allow --sdloramult to receive a list of multipliers
• LoRAs with multiplier != 0 would have fixed weights, as they are now
• LoRAs with multiplier 0 would be allowed runtime multiplier changes (through the sdapi and/or prompt). We could also add a parameter to gendefaults to still be able to set a default non-zero multiplier for them
• the presence of any customizable LoRA would force at_runtime mode. This way, we could keep the free_params_buffer call as-is, so setups with no customizable LoRAs would keep working as they are now, with no extra memory usage. It may even be possible to force at_runtime only for the customizable LoRAs.

What do you think?

By the way, it's also possible to support the <lora:name:weight> syntax as an UI functionality, parsing and converting it to the sdapi parameter on the stable-ui. I'm not exactly looking forward to implement it that way, but it would make sense from a compatibility POV, since it'd allow LoRA loading from other sdapi servers too.